Tinkering Under The Hood: Interactive Zero-Shot Learning with Pictorial Classifiers
نویسندگان
چکیده
We consider the task of visual zero-shot learning, in which a system must learn to recognize concepts omitted from the training set. While most prior work make use of linguistic cues to do this, we do so by using a pictorial language representation of the training set, implicitly learned by a CNN, to generalize to new classes. We first demonstrate the robustness of pictorial language classifiers (PLCs) by applying them in a weakly supervised manner: labeling unlabeled concepts for visual classes present in the training data. Specifically we show that a PLC built on top of a CNN trained for ImageNet classification can localize humans in Graz-02 and determine the pose of birds in PASCAL-VOC without extra labeled data or additional training. We then apply PLCs in an interactive zero-shot manner, demonstrating that pictorial languages are expressive enough to detect a set of visual classes in MSCOCO that never appear in the ImageNet training set.
منابع مشابه
Tinkering Under the Hood: Interactive Zero-Shot Learning with Net Surgery
We consider the task of visual net surgery, in which a CNN can be reconfigured without extra data to recognize novel concepts that may be omitted from the training set. While most prior work make use of linguistic cues for such ”zero-shot” learning, we do so by using a pictorial language representation of the training set, implicitly learned by a CNN, to generalize to new classes. To this end, ...
متن کاملLink the Head to the "Beak": Zero Shot Learning from Noisy Text Description at Part Precision
In this paper, we study learning visual classifiers from unstructured text descriptions at part precision with no training images. We propose a learning framework that is able to connect text terms to its relevant parts and suppress connections to non-visual text terms without any part-text annotations. For instance, this learning process enables terms like “beak” to be sparsely linked to the v...
متن کاملProbabilistic Zero-shot Classification with Semantic Rankings
In this paper we propose a non-metric rankingbased representation of semantic similarity that allows natural aggregation of semantic information from multiple heterogeneous sources. We apply the ranking-based representation to zeroshot learning problems, and present deterministic and probabilistic zero-shot classifiers which can be built from pre-trained classifiers without retraining. We demon...
متن کاملOne-Shot Reinforcement Learning for Robot Navigation with Interactive Replay
Recently, model-free reinforcement learning algorithms have been shown to solve challenging problems by learning from extensive interaction with the environment. A significant issue with transferring this success to the robotics domain is that interaction with the real world is costly, but training on limited experience is prone to overfitting. We present a method for learning to navigate, to a...
متن کاملZero-Shot Learning to Manage a Large Number of Place-Specific Compressive Change Classifiers
With recent progress in large-scale map maintenance and long-term map learning, the task of change detection on a large-scale map from a visual image captured by a mobile robot has become a problem of increasing criticality. Previous approaches for change detection are typically based on image differencing and require the memorization of a prohibitively large number of mapped images in the abov...
متن کامل